Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Format and lint notebooks #817

Merged
merged 6 commits into from
Sep 23, 2024
Merged

Format and lint notebooks #817

merged 6 commits into from
Sep 23, 2024

Conversation

itcarroll
Copy link
Collaborator

@itcarroll itcarroll commented Sep 18, 2024 •
edited
Loading

Closes #788

Re-enable the format and lint tool for notebook (.ipynb) files everywhere in the repo (which turns out to be in notebooks, docs/howto, and mostly docs/tutorials).

The only lint rules I found to be not great for notebooks are T201 and T203, which are prohibitions on print and pprint in production code. I ignored them for "*.ipynb" in [tool.ruff.lint.per-file-ignores].

I found two packages missing from [project.optional-dependencies.docs] that are needed for the notebooks to run, and added them with a minimum dependency taken from the version pip would install. uv add --extra docs dask cftime.

I did not attempt to fix some remaining errors in the notebooks that are beyond the scope of linting. They did not all run when I started and still don't all run (but not because of missing packages!).

Pull Request (PR) draft checklist - click to expand
  • Please review our
    contributing documentation
    before getting started.
  • Ensure an issue exists representing the problem being solved in this PR.
  • Populate a descriptive title. For example, instead of "Updated README.md", use a
    title such as "Add testing details to the contributor section of the README".
  • Populate the body of the pull request with:
  • Update CHANGELOG.md with details about your change in a section titled
    ## Unreleased. If such a section does not exist, please create one. Follow
    Common Changelog for your additions.
  • Update the documentation and/or the README.md with details of changes to the
    earthaccess interface, if any. Consider new environment variables, function names,
    decorators, etc.

Click the "Ready for review" button at the bottom of the "Conversation" tab in GitHub
once these requirements are fulfilled. Don't worry if you see any test failures in
GitHub at this point!

Pull Request (PR) merge checklist - click to expand

Please do your best to complete these requirements! If you need help with any of these
requirements, you can ping the @nsidc/earthaccess-support team in a comment and we
will help you out!

  • Add unit tests for any new features.
  • Apply formatting and linting autofixes. You can add a GitHub comment in this Pull
    Request containing "pre-commit.ci autofix" to automate this.
  • Ensure all automated PR checks (seen at the bottom of the "conversation" tab) pass.
  • Get at least one approving review.

📚 Documentation preview 📚: https://earthaccess--817.org.readthedocs.build/en/817/

@codecov-commenter
Copy link

codecov-commenter commented Sep 18, 2024 •
edited
Loading

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (main@68e19a2). Learn more about missing BASE report.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main     #817   +/-   ##
=======================================
  Coverage        ?   73.88%           
=======================================
  Files           ?       31           
  Lines           ?     2003           
  Branches        ?        0           
=======================================
  Hits            ?     1480           
  Misses          ?      523           
  Partials        ?        0

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@itcarroll
Copy link
Collaborator Author

itcarroll commented Sep 18, 2024 •
edited
Loading 

uv-lock..................................................................Failed
- hook id: uv-lock
- exit code: 2

@MattF-NSIDC (use mfisher87) I'm unable to find an explanation for what to do about pre-commit's uv-lock check, which is no doubt failing because I've added packages to pyproject.toml. Can you explain how to safely update uv.lock?

@mfisher87
Copy link
Collaborator

Hey @itcarroll for future reference, I don't log in to @MattF-NSIDC often so I don't see the notifications sent there :)

Did the failing check not also update uv.lock when it failed? This was the output for me when I added flask to the dependencies array in pyproject.toml:

$ pre-commit run -a uv-lock
uv-lock..................................................................Failed
- hook id: uv-lock
- files were modified by this hook

warning: `VIRTUAL_ENV=/home/robatt/.cache/pre-commit/repo16nnfz_r/py_env-python3.12` does not match the project environment path `.venv` and will be ignored
Using Python 3.12.5 interpreter at: /home/robatt/.local/share/miniforge3/envs/earthaccess/bin/python3
Resolved 222 packages in 1.12s
Added blinker v1.8.2
Added flask v3.0.3
Added itsdangerous v2.2.0
Added werkzeug v3.0.4

$ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   pyproject.toml
	modified:   uv.lock

@mfisher87
Copy link
Collaborator

Oh, and thanks so much for taking on this important piece of technical debt!! 🙇

@itcarroll
Copy link
Collaborator Author

Understand now that the uv-lock hook is throwing an error, so that's why it's not updating the uv.lock file.

The error I could not make any sense of was

(earthaccess) $ pre-commit run uv-lock
warning: `VIRTUAL_ENV=/Users/icarroll/.cache/pre-commit/repo48a95whq/py_env-python3` does not match the project environment path `.venv` and will be ignored
error: Failed to build: `earthaccess @ file:///Users/icarroll/tmp/earthaccess`
  Caused by: Failed to deserialize cache entry
  Caused by: wrong msgpack marker FixArray(2)

I tried reinstalling everything. What eventually made the error go away was updating the rev on the uv-pre-commit repo to "0.4.12". Is that okay @mfisher87?

@itcarroll itcarroll marked this pull request as ready for review September 18, 2024 17:59
@itcarroll itcarroll requested a review from mfisher87 September 18, 2024 18:08
@mfisher87
Copy link
Collaborator

mfisher87 commented Sep 21, 2024 •
edited
Loading

I tried reinstalling everything. What eventually made the error go away was updating the rev on the uv-pre-commit repo to "0.4.12". Is that okay @mfisher87?

That's great! Thanks so much for tracking down a fix. That's a really weird error message, I don't know what to make of it 😆

I have some family in town so I can't give a full review until perhaps in the middle of the coming week!

@itcarroll itcarroll requested review from danielfromearth and removed request for mfisher87 September 23, 2024 15:29
chuckwondo
chuckwondo previously approved these changes Sep 23, 2024
View reviewed changes
Copy link
Collaborator

@chuckwondo chuckwondo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@itcarroll
Copy link
Collaborator Author

itcarroll commented Sep 23, 2024 •
edited
Loading

pre-commit.ci autofix

@itcarroll itcarroll force-pushed the format-and-lint-notebooks branch from 6ece411 to ea752f8 Compare September 23, 2024 18:13
@chuckwondo chuckwondo merged commit 4aa1223 into nsidc:main Sep 23, 2024
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chuckwondo chuckwondo chuckwondo approved these changes

@danielfromearth danielfromearth Awaiting requested review from danielfromearth

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Lint and format Jupyter Notebooks
4 participants
@itcarroll @codecov-commenter @mfisher87 @chuckwondo